PVM: Parallel View Maintenance under Concurrent Data Updates of Distributed Sources

Data warehouses (DW) are built by gathering information from distributed information sources (ISs) and integrating it into one customized repository. In recent years, work has begun to address the problem of view maintenance of DWs under concurrent data updates of diierent ISs. Popular solutions such as ECA and Strobe achieve such concurrent maintenance however with the requirement of quiescence of the ISs. More recently, the SWEEP solution releases this quiescence requirement using a local compensation strategy that now processes all update messages in a sequential manner. To optimize upon this sequential processing, we have developed a parallel view maintenance algorithm, called PVM, that incorporates all beneets of previous maintenance approaches while ooering improved performance due to parallelism. In order to perform parallel view maintenance, we have identiied two critical issues: (1) detecting maintenance-concurrent data updates in a parallel mode, and (2) correcting the problem that the DW commit order may not correspond to the DW update processing order due to parallel maintenance handling. In this work, we provide solutions to both issues. Given a modular component-based system architecture, we insert a middle-layer timestamp assignment module for detecting maintenance-concurrent data updates without requiring any global clock synchronization. In addition, we introduce the negative counter concept as a simple yet elegant solution to solve the problem of variant orders of committing eeects of data updates to the DW. We have proven the correctness of PVM to guarantee that our strategy indeed generates the correct nal DW state. We have implemented both SWEEP and PVM in our EVE data warehousing system. Our performance study demonstrates that a multi-fold performance improvement is achieved by PVM over SWEEP.